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Abstract 

We review with a tutorial scope the information theory foundations of 
quantum statistical physics. Only a small proportion of the variables that 
characterize a system at the microscopic scale can be controlled, for both 
practical and theoretical reasons, and a probabilistic description involv- 
ing the observers is required. The criterion of maximum von Neumann 
entropy is then used for making reasonable inferences. It means that no 
spurious information is introduced besides the known data. Its outcomes 
can be given a direct justification based on the principle of indifference 
of Laplace. We introduce the concept of relevant entropy associated with 
some set of relevant variables; it characterizes the information that is 
missing at the microscopic level when only these variables are known. For 
equilibrium problems, the relevant variables are the conserved ones, and 
the Second Law is recovered as a second step of the inference process. For 
non-equilibrium problems, the increase of the relevant entropy expresses 
an irretrievable loss of information from the relevant variables towards 
the irrelevant ones. Two examples illustrate the flexibility of the choice 
of relevant variables and the multiplicity of the associated entropies: the 
thermodynamic entropy (satisfying the Clausius-Duhem inequality) and 
the Boltzmann entropy (satisfying the _ff-theorem). The identification 
of entropy with missing information is also supported by the paradox of 
Maxwell's demon. Spin-echo experiments show that irreversibility itself 
is not an absolute concept: use of hidden information may overcome the 
arrow of time. 

keywords: quantum probabilities, inference, reduced description, relevant 
entropies, irreversibility paradox. 
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1 Probabilities in theoretical physics 



The purpose of this contribution is to present an overview of the approach to 
statistical mechanics bascxl on information theory. The concept of information, 
intimately connected with that of probability, gives indeed insight on questions 
of statistical mechanics such as the meaning of irreversibility. This concept was 
introduced in statistical physics by Brillouin (1956) and Jaynes (1957) soon 
after its discovery by Shannon in 1948 (Shannon and Weaver, 1949). An im- 
mense literature has since then been published, ranging from research articles 
to textbooks. The variety of topics that belong to this field of science makes 
it impossible to give here a bibliography, and special searches are necessary for 
deepening the understanding of one or another aspect. For tutorial introduc- 
tions, somewhat more detailed than the present one, see R. Balian (1991-92; 
2004). 

The meaning of probabilities, on which the concept of information relies, 
has long been a subject of controversy (Cox, 1946). They were introduced 
in the theory of games as frequencies of occurrence, directly related to the 
counting of configurations, and could be viewed in this framework as properties 
of the system in itself. This objective interpretation contrasts with the so-called 
subjective interpretation, which was initiated by Bayes and Laplace, and later 
on advocated by Jaynes (1957) and de Finetti (1974). 

While the mathematicians who focus on the structure of the theory of proba- 
bilities do not adopt a definite position on their objective or subjective meaning, 
the physicists who apply them should worry about their interpretation. Most 
physicists have reached a spontaneous, often implicit, philosophical position in- 
spired by their daily practice, which combines subjectivism and materialism. 
During the XlXth century, science was generally considered as a discovery of 
Laws of nature existing outside us; in such an activity the observer can be dis- 
regarded once the exploration stage is over. The development of microphysics 
during the XXth century has led the physicists to lay more emphasis on the role 
of the observer. Theory is generally regarded as the construction of a partly 
imperfect image of the external world in our minds, which however becomes 
less and less blurred, more and more faithful as science progresses. Mathemat- 
ics provide the precise language which allows us to build this image; among 
mathematics, probabilities are the tool on which we rely to make quantitative 
predictions in a consistent and rational way, starting from the available infor- 
mation, in spite of various uncertainties. It is remarkable that the introduction 
of probabilities in our most fundamental theories has not prevented physics to 
become more and more efficient. 

The existence of barriers between the reality and us, which cannot be over- 
come at least at the present stage of physics, has become manifest in two differ- 
ent theoretical frameworks. On the one hand, the unification and the simplicity 
of the Laws of microphysics have led to a considerable development of statistical 
mechanics. It has become clear that all properties of materials at the macro- 
scopic scale, whether mechanical, thermal or electromagnetic, can in principle 
be explained by starting from their microscopic structure. Macroscopic theories 
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are then reduced to a phcnomcnological status. Even the Laws of thermodynam- 
ics have now lost their status of fundamental science since they can be derived 
from microphysics. However, describing fully a macroscopic object in terms 
of its microscopic constituents would require to deal with an inaccessibly large 
number of degrees of freedom. Neither can we control them experimentally nor 
even can we handle them numerically. Our only issue is a probabilistic treat- 
ment, where probabilities account for our lack of knowledge about, for instance, 
the positions and velocities of the molecules in a classical gas. 

On the other hand, microscopic systems are governed by quantum mechan- 
ics. There, physical quantities are mathematically represented as recalled in 
section 2 by elements of a non-commutative algebra. This feature implies, in 
particular, Heisenberg's inequality which expresses that two non-commuting 
variables (such as the position and the momentum of a particle, the three com- 
ponents of its angular momentum, the electromagnetic field and the number 
of photons, or the electric and magnetic fields at the same point) necessarily 
display statistical fluctuations: The values of such variables cannot be specified 
simultaneously. Here we need probabilities not only for practical reasons as in 
classical statistical mechanics, but because the fundamental theory itself im- 
plies intrinsic fluctuations. It is not simply the values of the physical variables 
which are incompletely known, but it is the very concept of physical quantities 
which, as a matter of principle, makes them simultaneously inaccessible. Their 
non-commutative nature forbids us to imagine that we might fully determine 
them. It is impossible to assume that, underlying our probabilistic description, 
the state of a microscopic system at a given time could be characterized by the 
values of its full set of physical variables. This impossibility, which contrasts 
with the use of probabilities in the description of systems in classical statisti- 
cal mechanics, is exemplified by Bell's inequalities and by the GHZ paradox 
(Greenberger et al, 1990) that we shall briefly review at the end of section 2. 

In both cases, and a fortiori in quantum statistical mechanics, we are led 
to always treat physical quantities as random. The probabilities that govern 
them occur either for practical reasons because we cannot hope to describe in 
full detail a macroscopic system, or for theoretical reasons because quantum 
fluctuations prevent us from thinking that the whole set of physical quantities 
needed to describe a system in the quantum formalism may take well-defined 
values. Thus the probabilities of quantum statistical physics cannot be regarded 
as properties of a system as such, but they characterize the knowledge about 
this system of its observers in the considered conditions. The probability law 
adequately describing the system depends on the information available to the 
observer or on the number of variables that he may control. Probabilities there- 
fore appear as having a partly subjective nature, on rather inter-subjective since 
two observers placed in the same conditions will assign the same probability law 
to a system. 

Moreover, such a probabilistic description does not refer to a single object. 
Explicitly or implicitly we regard this object as belonging to a statistical ensem- 
ble of similar objects, all prepared under similar conditions and characterized 
by the same set of given data. This ensemble may be real, if we deal with 
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predictions about repeated experiments, or gedanken, if we deal with a single 
event. 

Even though quantum mechanics requires the use of probabilities to describe 
systems at the microscopic scale, and even though these probabilities charac- 
terize the knowledge of observers about such systems, the theory also displays 
objective features that depend on the systems only. Indeed, as we shall recall 
below, the microscopic equations of motion of an isolated system are fully deter- 
ministic since the Hamiltonian operator is exactly known. (For most properties 
of atoms, molecules and macroscopic materials, only the very simple kinetic 
energy and electromagnetic interactions of the elementary constituents, the nu- 
clei and the electrons, are relevant.) Whereas at each time the statistics of the 
physical quantities are accounted for by a probability law, the evolution of this 
law is governed at the microscopic scale by the reversible equation (01 below, 
which depends only on the object studied. The probability distribution, which 
characterizes our knowledge, is transferred from one time to another in a deter- 
ministic way - unless we drop information or unless part of the Hamiltonian is 
ill-known. 

Other objective features can also emerge at the macroscopic scale. Owing to 
the large number of constituents, the probability distribution for many macro- 
scopic variables can be sharply peaked. The uncertainties of the observers and 
the subjective aspect of probabilities can then be disregarded, and statistical 
predictions are replaced by objective assertions, which hold even for a single 
object. 

The first stage of an inference consists in assigning to the system a prob- 
ability distribution that accounts for the available data but is otherwise unbi- 
ased. Then, predictions are derived from this distribution by means of stan- 
dard techniques of statistical physics. For instance, we may wish to predict 
the two-particle correlation function in a simple liquid at equilibrium, so as to 
understand how it scatters light; in this problem the data which characterize 
the macroscopic equilibrium state are the energy and particle number per unit 
volume. This correlation function can be deduced from the density in phase, 
that is, the probability distribution of the N particles of the liquid in the 6A^- 
dimensional phase space. However a preliminary question should be solved: 
From the sole knowledge of the energy and the particle number, how should we 
reasonably choose this probability distribution? 

2 The formalism of quantum (statistical) me- 
chanics 

We shall work within the framework of quantum statistical mechanics (Thirring, 
1981, 1983; Balian, 1989), which is conceptually somewhat simpler than clas- 
sical statistical mechanics. To avoid mathematical complications, we consider 
finite systems only and assume that the thermodynamic limit for extensive sys- 
tems is taken in the end. The discreteness of spectra is then a feature which 
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allows to by-pass some difficulties arising in classical statistical mechanics from 
the continuity of the variables. For instance, the classical limit of quantum sta- 
tistical mechanics generates the suitable measure in the 6A/'-dimensional phase 
space; this measure includes a factor issued from the Pauli principle for in- 
distinguishable particles, which ensures the extensivity of entropy and solves the 
Gibbs paradox. These facts can be explained in the framework of classical sta- 
tistical mechanics (van Kampcn, 1984). but less automatically. Obtaining the 
Third Law of thermodynamics as a consequence of statistical mechanics also 
requires quantization, since the behaviour at low temperatures of the entropy 
of a macroscopic system is governed by the low energy behaviour of its level 
density. 

The mathematical description of a physical system in quantum mechanics 

involves a Hilbcrt space, and the physical quantities arc represented by the 
Hermitean operators A in this space. As indicated in the introduction, these 
operators, termed observables, play the role of random variables but constitute 
a non-commutative algebra. For instance, the three components of the spin ^ 
of a fermion are described in a 2-dimensional Hilbert space by the Pauli opera- 
tors &x, ^y, which are characterized by the algebra UiUj = iJ2k ^ijk^k] for 
a particle on a line, the algebra is generated by the position and momentum 
operators x and p in the Hilbert space of wavefunctions, with [x^p] = ih. The 
specific features of quantum physics, compared to the standard probability the- 
ory or to the classical statistical mechanics, lie in this non-commutation of the 
observables. 

In quantum mechanics, the "state of a system" , whatever formalism is used 
to characterize it (wavcfunction, state vector, density matrix, etc), is an ir- 
reducibly probabilistic concept. As stressed in the introduction, one cannot 
imagine the existence of an underlying, purely objective description in terms of 
the values of all the physical quantities attached to the system. We are led to 
adhere to the Bayesian or Laplacian subjective conception of probabilities, and 
to regard the "state" as the best possible description of the system, which allows 
us to make any possible probabilistic prediction about it - or rather about the 
statistical ensemble to which it belongs. 

A set of probabilities is equivalent to the collection of expectation values 
that they allow us to evaluate. A quantum state is thus characterized by the 

correspondence A i-^ {^y* ^ ^ which associates with any observable A its ex- 
pectation value A in the considered situation. This correspondence has a few 
natural properties. The hermiticity of A entails that A is real (quantum me- 
chanics involves complex numbers, but physical predictions yield real numbers). 

For the unit operator, ^/^ = 1. The correspondence is linear; in fact, the lin- 
earity l^A + = (^A^ + ^-^^ f'-"' P^^'' commuting observables A and 

B is sufficient to imply linearity for the whole set of observables (provided the 
dimension of the Hilbert space is larger than 2: Gleason's theorem). Finally 

the statistical fluctuation of a physical quantity is expressed as \A^ \ — (A) 
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in terms of the expectation values Imposing that i^A'^J is non- negative for 

any Hermitean A expresses that a variance cannot be negative. 

For finite systems these properties are implemented through the existence 
of a density operator D in Hilbert space which represents the state. The above 
correspondence from observables to their expectation values is expressed as 

(^Aj = A = TtAD , (1) 

where D is a Hermitean positive operator with unit trace. The density operator 
thus gathers our whole probabilistic information about the full set of observables 
of the system. It plays with respect to observables the same role as a usual 
probability distribution does with respect to random variables. A wavefunction 
or a state vector j^*) appears as nothing but a special case of density operator; 
in this case D = l^") (\E'| reduces to a projection on the considered pure state 
and the expectation values Q read A = (^l^l^*). Note that, due to the 
irreducibly probabilistic nature of the quantum theory, there is no conceptual 
difference between "quantum mechanics" and "quantum statistical mechanics" , 
since a wavefunction is nothing but a means for evaluating probabilities. 

For time-dependent problems, we need in order to make predictions to ex- 
press how this information given at some time, is transformed at later times. 
This is expressed in the Schrodinger picture by letting D depend on time while 
the observables A remain fixed. For an isolated system, with given Hamiltonian 
H, the evolution of D (t) is governed by the Liouville~von Neumann equation 
of motion: 

(2) 

Whereas the definition of quantum states involves both the system and the 
observers and has a subjective aspect because the density operator appears 
as a tool to make consistent predictions from the available information, the 
evolution refers to the system only. This purely objective nature of the evolution 
is made clear in the alternative Heisenberg picture, where the density operator 
D remains fixed while the observables change in time according to Heisenberg's 
equation of motion 

(3) 

which is deterministic and reversible. Equations (O and Q are equivalent as 
regards the evolution of the expectation values Heisenberg's equation ||2J) 
exhibits better the fact that the dynamics is a property of the considered system, 
since it deals with the observables which represent mathematically the physical 
quantities belonging to this system, independently of observation. In fact, eq. 
^ simply expresses that the operator algebra remains unchanged in time while 
the various observables A evolve; this property is equivalent to the unitary of 
the evolution, with the Hamiltonian as generator of the unitary motion. As 
another advantage, Heisenberg's picture allows us to evaluate correlations be- 
tween observables of the same system taken at different times. However, because 
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we focus below on information, we shall start from Schrodinger's picture rather 
than Heisenberg's. When expressed by means of 10) instead of Q, the evolution 
takes a subjective aspect, since Q describes the transfer of information from 
some observables to other ones generated by a completely known dynamics. We 
can then modify eq. l(2Jl so as to account for losses of information that may take 
place during a dynamical process. 

In the classical limit, observables A are replaced by commuting random vari- 
ables, which are functions of the positions and momenta of the N particles. Den- 
sity operators D are replaced by probability densities D in the 6A'^-dimensional 
phase space, and the trace in by an integration over this space. The evolution 
of D is governed by the Liouville equation. 

To conclude this section, we show how the irreducibly probabilistic nature of 
quantum states follows from the non-commutation of the observables. Consider 
first two physical quantities represented by two observables A and B which do 



not commute, 



A,B 



2iC. Since for any A the operator (^A + iXB^ {^A — iXB 
can be regarded as the square of a Hermitean operator (as obvious by diago- 
nalization), its expectation value is non- negative, which implies (A^\(B'^\ > 



-\2 

C ) . This Heisenberg's inequality sets a lower bound to statistical fluctua- 
tions (for instance, 2(7 = h ioi A ^ x, B ^ p yields AxAp > h/2). Accordingly, 
non-commuting physical quantities are incompatible: They cannot be measured 
nor even specified simultaneously. 

Bell's inequalities and the GHZ paradox (Greenberger et al, 1990) also arise 
from non-commutation. Consider, for instance, three spins m = 1, 2, 

3; define the observables i^^^ = u'^^\ B^^'> = u^^^uf \ (7^1) = i^^^S^i), and 
B^"), C^^") by cychc permutation. All of them have -1-1 and -1 as 
eigenvalues, the A's commute with one another as well as the B's and A*^™) 
commutes with B^™\ For m =^ n, A^™^ and B^") anticommute, so that the 
three observables (7*^"^ commute with one another. Hence, we can imagine 
that the 3-spin system is prepared in a pure state j^*) which is the common 
eigenvector of (7*^^\ (7*^^^ and (7^^^ characterized by their eigenvalues c^^\ c^^\ 
c(3) all equal to +1. In this state, the fact that C^^'> = A^^^B^^) takes the 
value c^^^ = +1 implies that a'-^-' = fe^^-*; more precisely, if we were to mea- 
sure A^^^ and B^^^ simultaneously (they commute), we would find for them 
either the values a*^^^ = 6'^-' ~ +1 or the values a*^^^ — b^^^ = —1 (with equal 
probabilities since the expectation value of A^^^ or B^^^ vanishes). Likewise 
we can assert the complete correlations a*^^-' — b'^^^ and a'^^^ = b^'^\ How- 
ever the above definitions and algebraic relations imply the operator identities 
5(1)^(2)5(3) ^ J and C'(i)(7(2)(7(3) = -i(i)i(2)i(3). Hence, although the 
three statements a^^^^ = b^^\ a^^' — U"^^ and a'^^^ = (in the above sense) are 
separately true, they cannot be true together: Since the product i(i)i(2)^(3) 
takes the value — c'-^-'c^^-'c'-^-' = — 1 in the considered state |^'), the simultane- 
ous measurement of A'^\ A^"^^ and A'^'^^ (which commute) is expected to yield 
values 0*^^^ a^^^ and a'^^^ equal to -1-1 or —1, but necessarily with a product 
a'^'a'^^-'a'^^ ~ —1, in contradiction to the naive prediction a'^^a^^^a'^^ = +1 
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which would result from a^^) = b^'^\ a^^) = 5(2), a^3) ^ ^(3)^ ^(1)^(2)^(3) ^ 
This has been experimentally confirmed. Thus, everyday's logics is violated. 

The only issue is to regard the "state" not as a property of the system alone, 
but as a probabilistic means for prediction. The current expression "the state 
of the system" that we used (and that we may still use for convenience below) 
is improper; it is meant as "the probability distribution (for non-commuting 
physical quantities) which allows us to predict the properties of the statistical 
ensemble in which the system is embedded" . Indeed, we can safely predict that 
a measurement of ^^^^ and i?^^^ will give a*^^^ = 6*^^^ for any element of the 
ensemble, and likewise for A^^) ^nd i?*^^)^ However, such measurements must 
be performed with different apparatuses on different samples (described by the 
same state vector l^*)), because A^^^ and B^^) do not commute and hence cannot 
be measured nor even specified simultaneously. We cannot find in the ensemble 
any system for which takes with certainty, say, the value a^^^ = +1 and 
the value fc^^^ = +1: The assertion that both a^^^ = b^^^ and a^^) ^ 6^^) 
is meaningless. The contradiction existing between the equalities a^^-* = 6^^'' , 
a(2) = 6(2), a(3) = 6(3), 6(1)5(2)5(3) ^ ^ ^nd a(i)a(2)o(3) = -1 implies that we 
should not regard the two correlations a''^^ = and a(2-' = b^"^^ as intrinsic 
properties of the system, but rather consider each one as an exact prediction 
about a specific measurement^ namely that of A^^^ and B''^^ in one case, that 
of ^(2) and B(2) in the other case, the measurements of ^(^^ and 5(2) being 
incompatible. The state vector \^) itself, which synthesizes all such information, 
does not describe intrinsic properties of the systems belonging to the considered 
ensemble, but rather tells us how they would behave in some experiment or 
another. Each assertion such as a(i) = 6(1) (which to be tested would require 
an interaction of a system with an apparatus measuring A^^^ and B^^^), is true 
only in a given context, which should be specified even if the measurement is not 
actually performed. The interpretation of quantum mechanics is thus tightly 
connected with measurement theory. The type of correlations that it involves 
cannot be accounted for in terms of hidden variables. 



3 The measure of uncertainty 

The knowledge embodied in a density operator D is always probabilistic, since 
D provides us only with the expectation values (0. At least some among the 
variances evaluated from ^ must be finite, due to non-commutation, because 
there exists no common eigenvector for the whole set of observables. It is there- 
fore natural to wonder whether one density operator is more informative than 
another (Thirring, 1983; Balian, 1991). To this aim we associate with each D a 
number, the von Neumann entropy 

S'vN (d) = -Trl) InLi , (4) 

which measures in dimensionless units our uncertainty when D summarizes our 
statistical knowledge on the system. This quantity is the quantum analogue 
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of Shannon's entropy — J2mPm I'^Pm,; the latter number measures the amount 
of information which is missing on average when we wait for the reception of 
some message belonging to a set {to}, if each message m is expected to occur 
with a probability Pm (Shannon and Weaver, 1949). However, here, there will 
be no reception of a message since we shall never observe fully the system at 
the microscopic scale. The quantity S (d) qualifies merely our present knowl- 



edge through D, without reference to future observations which can change our 
information. 



We may also interpret S ID \ as a measure of the disorder that we are facing 



when our information is characterized by D. This identification of the concept 
of disorder with a lack of knowledge may seem questionable, since disorder 
seems an objective property of the system whereas information is attached to 
the observer. However, even in everyday's life, we can recognize that disorder 
is a relative property. My desk may look quite disordered to a visitor, but 
it is for me in perfect order, if I am able to retrieve immediately any one of 
the many files that lie scattered on it. Likewise, a pack of cards fully mixed 
by a conjurer is disordered for the audience, but not for him, if he has kept 
knowledge of the ordering of the cards. Information theory allows us to make 
quantitative Maxwell's remark about the concept of order: "Confusion, like the 
correlative term order, is not a property of material things in themselves, but 
only in relation to the mind who perceives them" . 

Historically, Boltzmann was the first to identify, in many successive works, 
the entropy of thermodynamics with a functional of the distribution in phase 
space of classical statistical mechanics - even though the concept of probabil- 
ity was not yet fully settled. Von Neumann extended this idea to quantum 
physics by introducing the definition Q. The existence of these expressions 
inspired Shannon (1949) when he created the theory of communication, which 
is based on the possibility of assigning numbers to amounts of information. In 
this context, the subjective nature of entropy is manifest. From the fact that it 
measures the missing information associated with the probability distribution 
for the considered set of messages, it appeared that it was a general concept 
complementing probability. With this new interpretation as lack of informa- 
tion at the microscopic scale, the concept has returned to statistical physics 
(Brillouin, 1956; Jaynes, 1957), throwing new lights on entropy. 

The von Neumann entropy is characterized by many properties which con- 
firm its interpretation as a measure of uncertainty. It is additive for uncorrelated 
systems, subadditive for correlated ones (which means that suppressing correla- 
tions raises the uncertainty), concave (which means that putting together two 
different statistical ensembles for the same system produces a mixed ensem- 
ble which has a larger uncertainty than the average of the uncertainties of the 
original ensembles). The maximum InW of Q is obtained, for a finite Hilbert 
space with dimension W when D = I/W. Its minimum is obtained for pure 
states D, which are the least uncertain states that quantum mechanics allows, 
although they are still probabilistic. 
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4 Maximum entropy criterion 



The availabihty of the density operator Z? at a given time to aUows us to make 
any statistical prediction on the considered system (or more precisely on a sta- 
tistical ensemble to which it belongs), either at the same time through or 
at later times through However, a preliminary problem arises. During the 
preparation of the system, before the time to, only a small set of data are con- 
trolled. Let us denote as Ai the observables that are controlled, and as Ai their 
expectation values for the considered set of repeated experiments. From this 
partial information we wish to infer other quantities. In other words, we wish 
to assign to the system a density operator D, by relying on the sole knowledge 
of the set 

A^TtAD. (5) 

The maximum entropy criterion consists in selecting, among all the den- 
sity operators subject to the constraints ((SJ, the one, Dr, which renders the 
von Neumann entropy Q maximum (Jaynes, 1957). An intuitive justification 
is the following: for any other D compatible with jSJl, we have by construction 

(^Dj < S'vN (^r) ■ The choice of £)r thus ensures that our description 
involves no more information than the minimum needed to account for the only 
available information © . The difference S (-Dr,^ — S (^Dj measures some extra 

information included in D, but not in Z?r, and hence, not in the only known 
expectation values Ai. Selecting Z?r rather than any other D which satisfies 
((SJ is therefore the least biased choice, the one that allows the most reasonable 
predictions drawn from the known set Ai about other arbitrary quantities (0. 

We may also interpret the maximum entropy criterion as the choice of the 
most uncertain state among those which satisfy the constraints © imposed by 
the data Ai, or equivalently of the most disordered one. This criterion may 
be regarded as a generalization of Laplace's principle of insufficient reason (or 
Bayes's principle), which states that equal probabilities should be assigned to all 
possible events in case nothing is known about the occurrence of these events. 
Indeed, if the Hilbert space associated with the system has a finite dimension 
W, the entropy is largest when D is proportional to the unit matrix /, so that 
the criterion yields Db. = I/W, describing equiprobability, in case there are no 
constraints (0) on expectation values. 

This use of the lack of information as a tool for statistical inference has 
a weakness: it relies on the assumption that S'vN is the adequate measure of 
bias in statistical physics. Direct justifications of the outcome of the maximum 
entropy criterion, based on some requirements of consistency of the inference 
procedure, have therefore been given. They are discussed in detail by Uffink 
(1995). 

An alternative direct justification relies on the introduction of a "supersys- 
tem", made of a large number TV of mental copies a = 1, 2, . . . A/" of the system 
in hand. The statistical data are the same for all individual systems a, i.e., the 
observables Ai^a (a = 1,2,.. .TV) have the same expectation value Ai. This value 
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is identified with the vahie of the average observable J2a ^i,^ pertaining 
to the supersystem, which has weak statistical fluctuations (at most in A/""^/^, 
due to non-commutation of the observables Ai). Such an identification throws 
a bridge between the subjective and frequencial interpretations of probabilities. 
The maximum entropy criterion then arises merely as a consequence of the prin- 
ciple of insufficient reason applied to the supersystem. This idea was introduced 
by Gibbs in the case where the only datum (O is the energy: he showed that, 
for large Af, each system a is in canonical equilibrium if a microcanonical dis- 
tribution is assigned to the supersystem. The extension to quantum statistical 
mechanics, with non-commuting observables Ai, was worked out by Balian and 
Balazs (1987). 



5 Generalized Gibbsian distributions and rele- 
vant entropy 

For any given set of relevant observables Ai, the expectation values Ai of which 
are known, the maximum of the von Neumann entropy I0J under the constraints 
(0 is readily found by introducing Lagrangian multipliers, 7^ associated with 
each equation (O and ^' associated with the normalization of D. Its value, 

iSvN (^r) ' ^® reached for a density operator of the form 



Dr = exp 



where the multipliers are determined by 
TiDnA.^A, , 



TiDn = 1 



(6) 



(7) 



This least biased density operator has an exponential form, which generalizes the 
usual Gibbs distributions and which arises from the logarithm in the definition of 
the von Neumann entropy. The concavity of the von Neumann entropy ensures 
the unicity of Z?r. 

The equations ^ can conveniently be written by introducing a generalized 
thermodynamic potential ^E" (7,), defined as function of the other multipliers 7^ 

by 



^' (7i) = In Tr exp 



(8) 



The relations between the data Ai and the multipliers 7^ are then implemented 
as 

-A, . (9) 



5* 

97i 



The corresponding entropy S'vn (^r) = "^R (^*) ^ function of the variables 



11 



Ai (or equivalently 7,), found from ||SJ) and © as 



SniA,)^^ + J2l^A . (10) 



We term it the relevant entropy associated with the set Ai of relevant observables 
selected in the considered situation. By construction, it measures the amount 
of information which is missing when only the data Ai are available. Eq. H1(J|I 
exhibits (Ai) as the Legendre transform of 5* (7^). The relations ^ between 
the variables 7; and the variables Ai can therefore be inverted as 

dA-=^'- ^'^^ 

The following examples will show that the concept of relevant entropy en- 
compasses various types of entropies, introduced in different contexts. 



6 Equilibrium thermodynamics 

As stressed by Callen (1975) the Laws of thermodynamics do not deal with 
dynamics, but with the comparison of the initial and the final equilibrium states 
of a system. At the macroscopic level, an equilibrium state is characterized by 
the values Ai of a set of extensive conservative variables, such as energies or 
particle numbers. In fact, the thermodynamic laws also hold for metastable 
states which behave as stable states for small variations of the parameters. 
In such cases, the thermodynamic variables Ai include the nearly conserved 
quantities, which strictly speaking obey no conservation law but are conserved 
over the time scale of the considered process. This occurs, for instance, below a 
transition point for an order parameter, or when some a priori allowed chemical 
reaction does not take place under the prevailing conditions. The concept of 
equilibrium is thus relative: it depends on the efficiency of the interactions and 
many equilibria are not truly stable, but metastable over very long times. (As 
an extreme example, we may note that nuclear forces are effective only within 
each nucleus of a material; although strong, they are inhibited between one 
nucleus and another by the Coulomb repulsion. Nuclear equilibrium, which 
would imply that nuclear reactions have transformed both the light and the 
heavy nuclei through fusion or fission into iron nuclei, is never reached.) 

Callen's statement of the Second Law refers to an isolated, compound system, 
in which we include the possible sources of heat or work. Each subsystem is a 
homogeneous piece of extensive matter. In the data Ai that characterize the 
initial and the final state of the whole system, the index i denotes both the 
subsystem and the nature of the variable (energy, number of particles of each 
kind, volume). Various exchanges of such quantities may take place between the 
subsystems under the effect of interactions. In the initial state these exchanges 
are prohibited by some constraints, which express that the interactions between 
subsystems are dropped, so that the variables Ai can be specified independently. 
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Later on, some interactions are restored so that some constraints are released. 
We wish to know the values of the variables Ai in the final equilibrium state thus 
reached. The answer is given by the Second Law, which postulates the existence 
of the thermodynamic entropy Sth, a function of the variables Ai which has the 
following properties (Callen, 1975). It is additive, being a sum of contributions 
of all subsystems; for each subsystem, it is extensive and concave. The final 
values of the variables Ai are then found by looking for the maximum of S'th, 
under constraints imposed by the initial data through the conservation laws and 
by possible inhibitions that forbid some exchanges between the subsystems. 

This formulation of the foundations of thermodynamics is directly related to 
the approach to statistical mechanics of sections 4 and 5, based on information 
theory. It suffices to identify the thermodynamic extensive variables Ai with the 
expectation values (0) of the conserved observables Ai. More precisely, the Ai^s 
are the operators that commute with the Hamiltonian of the overall compound 
system when the interactions between the subsystems are dropped. The expec- 
tation values of the energies or of the particle numbers of the subsystems can 
then be frozen. The macroscopic equilibrium state, characterized by the data 
Ai, is identified microscopically with the most disordered state (jSJ accounting 
for these data. This surmise can be justified statistically: if no other variable 
than these conserved quantities Ai is controlled, reasonable predictions on any 
other quantity should rely on the least biased choice ©. Moreover the vari- 
ances of all the macroscopic variables are then weak (of relative order iV~^, when 
A'^ is the particle number); hence predictions about them can be made nearly 
with certainty, as we observe macroscopically, and they are not affected by a 
coarse-graining of © . The assignment of the density operator jnj to an equilib- 
rium state can also be justified dynamically through non-equilibrium statistical 
mechanics, by showing that the Hamiltonian evolution while keeping the 
variables Ai constant, leads to the coarse-grained density operator with maxi- 
mum disorder, as measured by the von Neumann entropy. 

We thus identify the macroscopic thermodynamic entropy S'th {Ai) with the 
microscopic relevant entropy S'j^(Ai), which was defined as the maximum of 

the von Neumann entropy SvN (J^^ under the constraints © on the thermody- 
namic variables Ai, frozen in the initial state. The properties of S'th postulated in 
thermodynamics are thereby found as consequences of statistical mechanics: its 
additivity arises from the factorization of the density operator J^Jl into contribu- 
tions associated with each subsystem; its concavity results from the concavity of 
S'vn; its extensivity can be proved for a wide class of interparticle interactions 
including the realistic interactions (Ruelle, 1969; Lieb, 1976; Thirring,1983). 
The explicit form of Sth {Ai) can also be derived from microphysics, while only 
its existence is asserted in thermodynamics. 

When applied to the final state, the maximum entropy criterion of statistical 
mechanics can be worked out in two steps. In the first step Svn is maximized 
for fixed arbitrary values of the variables Ai; in the second step the outcome, 
already identified with Sth {Ai), is maximized with respect to the variables Ai 
subject to constraints imposed by the conservation laws and by the possible 
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exchanges between subsystems. The Second Law then appears merely as the 
second step of this maximization, which leads to the largest disorder. 

Eqs. l|!?|l- Hll|l reduce to standard equations in thermodynamics. In particular 
the expression (|11|) of the multipliers % allows us to identify them with Callen's 
intensive variables of thermodynamics; for instance, the multiplier 7 associated 
with the energy of a subsystem is the corresponding inverse temperature. Also, 
^ is identified with a Massieu thermodynamic potential. Owing to the large 
number of constituents of thermodynamic system, the statistical fluctuations of 
the macroscopic variables are negligible, the probabilistic nature of statistical 
physics which underlies thermodynamics becomes invisible, so that thermody- 
namics appears as a deterministic theory. The thermodynamic entropy Sth, 
although it can be identified with a measure of our lack of information about 
the system at the microscopic scale, as well as the parameters 7 entering the 
density operator, reach an objective status and become meaningful for a single 
macroscopic system. 

7 Elimination of the irrelevant variables 

In non-equilibrium statistical mechanics, a central problem consists in predicting 
the values at the time t of some set of variables Ai from the knowledge of their 
values at the initial time to. The set of relevant observables Ai now contains 
not only the constants of the motion and the quantities controlled or observed 
experimentally, but also other macroscopic variables, chosen in such a way that 
the variables Ai will eventually be coupled only to one another within a good 
approximation. We shall return below to the choice of this set, which for the 
time being we leave unspecified. 

A general solution of this inference problem is provided by the projection 
method of Nakajima and Zwanzig, which we only sketch here; for a simple 
introduction, see Balian (1999). Since at the microscopic level the dynamics is 
generated by the Liouville-von Neumann equation ^ , the equations of motion 
of Ai (t) should be generated from (O and Wc thus need first to transform 
the initial conditions on Ai (to) into initial conditions on D{to). Since at the 
initial time to nothing else than the set Ai {to) is specified, the least biased 
choice for D (to) is given by the maximum entropy criterion, in the form ©, 

From this initial state D {to) = -DR(to)j one derives the state D {t) at an 
arbitrary time t by solving the equation of motion Q . The relevant variables 
Ai [t) are then found from D {t) through ||SJ), which formally solves our inference 
problem. We give below a more explicit expression for this answer. 

The von Neumann entropy associated with D (t) remains constant in 
time, as readily checked from the unitarity of the evolution ((5J). This means 
that, if one considers all possible observables^ no information is lost during the 
evolution. However, in general, D (t) does not keep the exponential form ©, 
which involves only the relevant observables Ai. Following the procedure of 
sections 4 and 5, we can evaluate the lack of information associated with the 
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knowledge of the variables Ai (t) only. It is measured by the relevant entropy 
(|10() , where the multipliers 7^ (t) arc now time-dependent and are expressed in 
terms of the set Ai (t) through ^ or . By construction, we have 



Sk [A, (t)] > 5vN [D (Oj = -^vN [D (to) 

= 5vN \Dr (to)] = Sn [A, (to)] 



(12) 



The fact that the relevant entropy is in general larger at the time t than at 
the initial time to means that a part of our initial information on the set Ai 
has leaked at the time t towards irrelevant variables. This loss of relevant 
information characterizes a dissipation in the evolution of the variables Ai (t). 

The construction of section 5 associates, at each time, a reduced density oper- 
ator £)r (t) with the set of relevant variables Ai (t). As regards these variables, 
i^R (t) is equivalent to D (t) : 



Tr (t) A^TtD (t) A, ^ A, (t) 



(13) 



but it has the maximum entropy form @ and thus does not retain information 
about the irrelevant variables. It is parameterized by the set of multipliers 
7i (t) , in one-to-one correspondence with the set of relevant variables Ai (t) . 
Regarding density operators as points in a vector space, we can visualize (fig. 
1) the correspondence from D (t) to Db, (t) as a projection onto the manifold R of 
reduced states ((HJ . (The space of states D can be endowed with a natural metric 
(Balian et al, 1986) defined by ds^ — —d^S = Tr dDdlnD; the correspondence 
D — > Z)r then appears as an orthogonal projection.) We thus consider in 
parallel two time-dependent descriptions of the system, the most detailed one 
by D (t) which accounts for all observables, and the less detailed one by (t), 
or equivalently by the set 7i (t) , which accounts for the variables Ai (t) only, and 
is sufficient for our purpose. 

Rather than following the motion of D {t) which involves all the complicated 
irrelevant variables, we wish to eliminate these variables, i.e., to follow the 
motion of Dr (t) on the surface R. We thus wish to project the Liouville- 
von Neumann trajectory of D (t) onto R. Once the operators D are regarded as 
elements of the vector space of states, it is natural to regard the operators A as 
elements of the dual vector space, the space of observables, the scalar product 
being defined by ^ and noted as 



(i;Z)) = Tr AD ^ (A 



D 



(14) 



Linear transformations in either vector spaces or |i| are represented by 
"superoperators" acting either on their right or their left side. In particular the 
equation of motion can be rewritten as 



dD 

— = CD 
dt 



(15) 



15 




Figure 1: The reduction of the description. A state D of quantum statistical physics 
is represented by a point in a vector space. Here the first two axes schematize the 
relevant variables Ai — Tr A,;Z3, regarded as coordinates of D, while the third axis 
stands for its many irrelevant coordinates. The trajectory of D (t) is governed by the 
Liouville-von Neumann equation @. The reduced state (t), equivalent to D (t) 
as regards the relevant variables {A^} but maximizing the von Neumann entropy, 
is the intersection of the plane Ai [t) = Tr AiD (represented here as a line) and 
the surface R parameterized by the set {7^} according to @, JSJl. The trajectory 
of [t) is obtained by projecting that of D [t) on R. It starts from the initial 
point Dji (to) = D (to) and is governed by eg. (1171 or its approximation 12H . The 
von Neumann entropy is constant along the line D (t) , whereas for Dyi [t] information 
is lost towards the irrelevant degrees of freedom. 



in terms of the Liouvillian superoperator which transforms D into CD 



H,D 



/ih . The projection from D to £)r = VD is implemented by means of 



the projection superoperator 

dD 



V 



where D^, expressed by ®, is regarded as a function of the set 7^ directly and 
through 5", and hence as a function of the set Ai through ifTTl) . A superoperator 

which is a tensor product A'S)B of operators acts on (7 as A Tr (^BC^ on the right 

side and as Tr (cA^ B on the left side. Applied on the right side, V leaves Db. 

and its first-order derivatives dDn/d^i or dD^/dAi invariant. Applied on the 
left side, it leaves the operators Ai and / invariant. We note the complementary 
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projection superoperator on the irrelevant subspace as Q = J — V where J 
is the unit superoperator. We can then replace ((TK)l . using l(TB|) . by coupled 
equations for the two projected states, I)r, = VD, which depends on time 
through D and P, and QD. The elimination of the irrelevant variables is then 
achieved, explicitly though formally, by eliminating QD. 
We thus find an integro-differential equation of the form 

^ = VCb^ + f dt'M (i, t') Dn (f) , (17) 

J to 

which involves a memory kernel 

M{t,t')^V{t)CW{t,t')Cr{t') (18) 

acting in the time-dependent relevant space. This kernel depends on the evolu- 
tion superoperator W in the irrelevant space, itself defined by 

W{t,t') = Q{t)5{t-t') . (19) 

Eq. (|17|l is equivalent to the equation of motion for the relevant variables 
A, {t) = {A,0 (t) ) = {A,;Dn (t) ), namely : 

^ = {A,;PCDj,) + / dt\A;M {t, t') Dn {t') ) , (20) 

or for their conjugate set ji {t). The first term describes the direct coupling 
between relevant observables. The second one, which results from their indirect 
coupling through the irrelevant ones, is a retarded, memory effect depending on 
the past history of the AiS. 

We have already shown that, between the times t^ and t, some information 
about the relevant variables is transferred towards the irrelevant ones, which 
keep memory of the initial conditions. This transfer of information is measured 
by H12I) . The dissipation, that is, the time-derivative of the relevant entropy, is 
obtained from ©, ©, 113 as 

^ = E (i) j' dt'{A;M {t, t') Dn (f) ) . (21) 

It is a retardation effect, associated with the history of the coupling with the 
irrelevant observables. (The first term of H17|) does not contribute.) 

Till now, we have kept the relevant observables Ai arbitrary. Although exact, 
the equation (|2()|l is in general unworkable due to retardation. If we wish to use 
it in practice, we should make a suitable choice of the set Ai. Suppose this 
choice implies that the memory time over which A4 {t, t') is sizeable is short 
compared to the characteristic times of the set Ai (t). This property occurs if 
H18|) involves very many irrelevant degrees of freedom which evolve rapidly and 
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interfere destructively. We can then replace Dr, (f) by (t) and <o by — oo in 
P7|l . In this approximation the evolution of the relevant variables is governed 
by a mere differential equation at the time t 

^ ^ VCDn + ICDn , (22) 
where the dissipative kernel IC is defined by 

IC{t)= I dt'M {t, t') . (23) 

In this short-memory approximation, the dissipation (j21(l reduces to 

^ = ^7.(i.;^i?R). (24) 

i 

Although S-s. (t) > S'r (^o), nothing prevents to be negative at some times 
for an arbitrary choice of the set Af, at such times, relevant information (or 
order) which had previously been transferred to the irrelevant set and memorized 
there is recovered. Such a phenomenon indicates that the relevant variables 
have not been properly selected since they are sensitive to memory stored in the 
discarded variables. 

However, if the short-memory approximation H22|l holds, the relevant infor- 
mation is irretrievably lost: the dissipation rate (|^ is positive. We understand 
this fact by noting that {t) depends only on Dr {t — At) if the time-scale 
At characterizing the variations of Dr or S'r in H22|l or ()24(l can be treated as 
infinitesimal, but is large compared to the memory time of A4 {t,t'); the much 
earlier initial condition Dr (to) is forgotten. Taking then t as the initial time in 
eq. H12|l . we find S'r (t) > Sr (t — At) as anticipated. This entropy production 
measures a continuous flow of order or of information from the relevant to the 
irrelevant variables, which is generated by the evolution of the system. 

Altogether, if we choose the set of relevant observables Ai in such a way 
that the memory time associated with the dynamics of the irrelevant variables 
through Ai {t, t') is much shorter than the time scales associated with the mo- 
tion of the relevant variables Ai (t), the latter variables evolve autonomously. 
They obey the Markovian equation H22|l . and the associated entropy H24|l in- 
creases. The quasi-instantaneous kernel /C accounts for the dissipative effects 
induced on the variables Ai (t) by the eliminated ones. An adequate choice of 
observables Ai , for which equations of motion of the form H22|l involving through 
/C a negligible memory time are valid within a given precision, is thus not left 
to us. It is imposed by the microscopic dynamics through the possibility of sep- 
arating the time scales of Ai (t) and A4 {t,t'). this criterion does not depend 
on the observers, but only on the microscopic Hamiltonian and the considered 
conditions. For finite systems, eq. (|22|l still keeps a subjective aspect even if 
the short-memory approximation is justified, because it deals with probabilities 
or equivalcntly with variables Ai (t) that are expectation values. However, for 
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large systems, when statistical fluctuations are negligible, the equations of mo- 
tion for the set Ai (t) reach an objective status and can describe an individual 
system, in the same way as the equilibrium properties in thermodynamics (end 
of section 6). In such a situation, the increase of relevant entropy measured by 
the dissipation rate (|24|l also becomes independent of the observers, although 
it is microscopically interpreted as a loss of information. It quantifies the fact 
that the irrelevant variables act only through the quasi-instantaneous kernel /C, 
and that nothing is stored in them that could flow back towards the relevant 
set within any reasonable delay. 

Many semi-phenomenological approaches can be recovered from microphysics 
through the above elimination of a set of irrelevant variables that produce no 
memory effects on the relevant ones. We discuss two examples in sections 8 and 
9. 

8 Thermodynamics of irreversible processes 

Section 6 was devoted to what is usually referred to as "thermodynamics" , but 
should rather be called "thermostaiics" since its Laws apply to the compari- 
son of initial and the final state, not to the process itself which leads from one 
to the other. Thermodynamics proper provides the general laws that govern 
the time-dependence of the macroscopic variables, in compound systems of the 
type described at the beginning of section 6. These laws pertain to the ther- 
modynamic regime, which must be sufficiently slow so that the variables Ai (t) 
that characterize the macroscopic state at each time are the same conservative 
local variables as in section 6. However, the subsystems may now be volume 
elements, treated as infinitesimal but sufficiently large so that each one remains 
nearly at equilibrium at any time. Thus the variables Ai include the energies 
of the subsystems (or the energy density pE (r) in each volume element) for 
thermal processes; they include the numbers of constitutive particles (or their 
local density pN (r)) for diffusive or chemical processes. They also include the 
density of momentum pp (r) in hydrodynamics. 

The equations of motion of thermodynamics couple the time-derivatives of 
the conservative variables Ai to their fluxes, which in turn are related to the 
gradients of the intensive variables (defined as in the end of section 6). They 
describe phenomena such as relaxation or transport, in physics, mechanics or 
chemistry. The time-scales associated with this set of macroscopic equations are 
large compared to the time-scales over which the subsystems reach equilibrium. 
Indeed, owing to their conservative nature, the variables Ai can change only if a 
transfer occurs from one subsystem to another, and the couplings which govern 
such transfers are weak compared to the interactions within each subsystem 
which ensure local equilibrium. 

Hence, in the thermodynamic regime, the relevant variables of microscopic 
statistical physics should be identified with the above thermodynamic variables 
Ai. In the projected equations (|20|l that they obey, there is a clear separation 
of time scales and the short-memory approximation H22|l. H23|l is valid. The 
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resulting coupled instantaneous equations for the set Ai can then be identified 
with the phenomenological equations of thermodynamics. 

Since the relevant observables Ai are the same as in section 6, their associated 
relevant entropy Sb. (Ai) is again identified with the thermodynamic entropy 
Sth (Ai); its partial derivatives are related to the local intensive variables such 
as the local temperature, chemical potential or hydrodynamic velocity. Its rate 
of change, given by statistical physics as (|24|l . is positive. This property is 
identified with the macroscopic Clausius-Duhem inequality dSth/dt > 0, which 
can therefore be interpreted microscopically as the fact that some amount of 
information, or some order attached to the conservative variables is continuously 
lost during a dissipative thermodynamic process towards the non-conservative 
variables. 



9 Boltzmann gas 

The first historic appearance of statistical mechanics was the kinetic theory 
of dilute gases. In Boltzmann's approach, the state of the gas is represented 
at each time by the density of particles / (r, p, t) in the 6-dimensional single- 
particle phase space. Its evolution is governed by the semi phenomenological 
Boltzmann equation 

§ + -^-Vr/ = , (25) 

ot rn 

the right side of which is the collision integral. It is local in space and time. 

Boltzmann's description should be regarded as macroscopic. Indeed, in or- 
der to describe at the microscopic scale a dilute gas, we should use the classical 
limit of quantum statistical mechanics. We indicated in section 2 that the 
probabilistic state is then represented by the density D in the GA'^-dimensional 
phase space, which encompasses not only / (r, p) but also the 2-particle, A^- 
particle correlation functions in the single-particle phase space. Their evolution 
is governed by the equations of the BBGKY hierarchy, which are equivalent 
to the Liouville equation (the classical limit of and which, in contrast to 
Boltzmann's equation H25|) . are invariant under time-reversal. Let us identify 
/ (r,p,t) with our relevant set of variables Ai, the index i standing here for a 
point r, p in phase space. Following the procedure of section 7, we can eliminate 
formally all the irrelevant variables, that is, all the correlations between parti- 
cles. The BBGKY hierarchy then gives rise to an integro-differential equation of 
the type H2()|l for / (r, p, t). However, in a dilute gas, the memory time entering 
the kernel W, which describes the variations in time of the correlations, is the 
duration of a collision, since correlations are created by two-particle collisions. 
This duration is much shorter than the time between two successive collisions 
of a particle, which governs the evolution of / (r, p, t). Likewise, the size of the 
particles is much smaller than the mean free path. The short-memory approxi- 
mation is justified by this neat separation of scales. It leads on the macroscopic 
time-scale to an instantaneous and local equation for / (r, p, t) of the type (|22() . 
which is identified with the Boltzmann equation (|25|l . 



20 



The relevant entropy 5'b associated with the reduced Boltzmann description, 
in terms of the single-particle density / (r, p, t), is readily found by noting that 
D^i factorizes as a product of identical contributions from each particle and that 
the classical limit of a trace is an integral over phase space. We then get at each 
time 

= (/) - h-^ J £v d^p / (r, p) [1 - In /i^ ^ (r, p)] . (26) 

We recognize that this single-particle entropy is directly related to Boltzmann's 
7?- functional, H = J f In/, within a sign and within additive and multiplica- 
tive constants. Thus, Boltzmann's H -theorem, which expresses that H does 
not increase when / evolves according to Boltzmann's equation H25|l . is now 
interpreted as a continuous transfer of information from the single-particle data 
/ (r, p) to correlations which are built up by each collision. In this process, 
S (D) remains constant, expressing that no information would have been lost if 
we had been able to deal with the full BBGKY hierarchy, including correlations 
between a very large number of particles. 

Boltzmann's reduced description in terms of / (r, p) is more detailed than the 
hydrodynamic or thermodynamic description of the gas in terms of the densities 
Pat (r), pp (r), pE (r) of the conserved quantities (particle number, momentum, 
energy) . It depends at each time on one function / of six variables rather than on 
five functions p of three variables, which are obtained from / (r, p) by integration 
over p with weights 1, p andp^/2m, respectively. Boltzmann's description thus 
remains valid in circumstances where the hydrodynamic description fails, such 
as shock waves and boundary layers where space variations are so rapid that 
local equilibrium does not exist, or very dilute gases in ballistic regimes where 
collisions are too rare to ensure thermalization. 

Accordingly, Boltzmann's entropy Sb defined by H2t)|) should not be confused 
with the thermodynamic entropy Sth^ which for a gas is a function of the p's 
given by the Sackur-Tetrode formula. Nor should the H -theorem for Sb be 
confused with the Second Law or with the Clausius-Duhem inequality for Sth- 
The latter is valid in the local equilibrium regime only (but for any material, in 
contrast to the iJ-theorem). For dilute gases in the ballistic regime, only 5'b is 
relevant, S'th is useless. 

Since Boltzmann's description is less reduced than the hydrodynamic one, 
Sb is in general smaller than 5th. Actually, maximization of H26(l under local 
constraints on pN, pp and ps provides for /(r,p) a Maxwellian form, i.e.. 
In / (r, p) is a linear function of p and p^ /2m describing local equilibrium. When 
a gas is enclosed in a vessel, its irreversible evolution, which is governed by 
1)25(1 . leads it in a first stage to a state close to local equilibrium; Boltzmann's 
entropy 5b increases and eventually reaches 5th. Information is thus lost, not 
only towards the correlations which are not observed, but also through the 
evolution of / (r, p) towards a maximum entropy Maxwellian form. In the 
second stage, after the local equilibrium regime is reached, the collision term 
in (|25|l is dominated by the small deviation of / (r, p) from this form. The 
Chapman-Enskog method, based on the elimination of this deviation (which 
is formally similar to the elimination of QDr that leads to 1(17(1 then to ((22(l ). 
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then provides the thermodynamic equations for the gas (Fourier and Navier- 
Stokes equations) as consequences of the Boltzmann equation. In this regime 
the two descriptions become equivalent; Sb and Sth, which remain nearly equal, 
both increase towards the global equilibrium entropy. The information which is 
being lost during this second process is the one associated with non- uniformity 
in space of the densities p. 

10 Irreversibility and the multiplicity of entro- 
pies 

The discussion of section 7 and the two examples of sections 8 and 9 eluci- 
date the paradox of irreversibility, stated at the end of the XlXth century by 
Loschmidt, Zermelo and Poincare: How can the reversible Liouville-von Neu- 
mann equations of motion, which underlie the behaviour of any system, give rise 
to the irreversible evolutions that we observe at our scale? The answer is based 
on information theory. Macroscopic objects are so complex at the microscopic 
scale that we can control or observe only an extremely tiny proportion of the 
entire set of variables. These experimental variables are part of a set Ai, the 
set of relevant variables, that we may theoretically handle (the thermodynamic 
densities p in section 8, the single-particle density / in section 9). Irreversibility 
is then associated with an irretrievable leak of information (or of order) from 
the set Ai towards the irrelevant variables, extremely numerous and completely 
out of reach. The time-derivative of the relevant entropy S'r (Ai) measures this 
rate of dissipation. 

The fact that order which is lost for us within the irrelevant degrees of free- 
dom never returns to the relevant set is a consequence of the occurrence of two 
time scales in the microscopic dynamics. The short memory-time of the ker- 
nel Ai in H20() is associated with the very large number of irrelevant variables 
and with the destructive interference of their evolutions. The longer time-scale 
which characterizes the motion of the relevant set Ai , including the experimen- 
tally accessible variables, is associated with some specificities of the dynamics: 
local conservation laws for the thermodynamic regime, large ratio between the 
interparticle distance and the range of forces for gases in the ballistic regime. 
This separation of time scales provides, within a good approximation, instanta- 
neous equations (|22|) for the relevant set which, in contrast to the microscopic 
equations, are non-linear and irreversible. It should be stressed that the short 
memory in the exact equation H20|l is sufficient to ensure dissipation, and that 
the huge number of irrelevant variables also implies that the Poincare recurrence 
time is large beyond any imagination. Moreover, we have seen that, although 
the underlying concepts of probability, expectation value and information have 
a subjective nature, the resulting properties for the variables Ai {t) and for the 
relevant entropy iSr (t) become objective, depending only on the system in hand 
when this system is macroscopic. 

Irreversibility, when analyzed at the microscopic scale, is a statistical phc- 
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nomcnon. Wc have seen that probabilities underlie the very idea of information, 
a loss of which is identified with dissipation. An individual trajectory in clas- 
sical mechanics, a time-dependent pure state in quantum mechanics, cannot 
display irreversibility, since they encompass all the degrees of freedom. Boltz- 
mann already recognized this fact in kinetic theory, when he stressed the idea 
that, even when one considers a single trajectory in the 6A/'-dimensional phase 
space, its initial point should be regarded as a typical configuration belonging 
to some ensemble of macroscopically similar configurations. The logarithm of 
the number of these configurations is the initial entropy. The final point of the 
trajectory is likewise a typical configuration of a much larger set, all elements of 
which describe equilibrium configurations. The larger size of this set provides a 
quantitative interpretation of the entropy increase. 

Regarding the relevant entropy as missing information has led us to assign to 
a given process several entropies, associated with different levels of description. 
For a gas we have thus characterized in section 9 the coarseness of the most 
detailed description, of the Boltzmann description and of the thermodynamic 
description by the von Neumann entropy, the Boltzmann entropy and the ther- 
modynamic entropy, respectively. In order to understand better the mechanism 
which gives rise to the irreversibility of the dynamics of a classical gas, we can in- 
troduce a set of nested reduced descriptions (Mayer and Mayer, 1977). We start 
from the complete description in terms of the density D in the SA'^-dimensional 
phase space, which obeys the Liouville equation. The n-th reduced description 
is defined by dropping all information contained in the correlations of more than 
n particles. Boltzmann's description is the first reduced description; the second 
one accounts for the two-particle correlations created by the collisions, but not 
for the higher order correlations; and so on. At each level, there are reduced 
equations of motion resulting from the BBGKY set. The associated hierarchy of 
relevant entropies Si = Sb, S2, ■ ■ ■ , Sn, ■ ■ ■ , Sn = S (D) satisfy the inequalities 

5eq > 5th > 5i > ^2 >•••> 5„ >...> 5(D) , (27) 

which express an increasing content of information. Their time-dependence, 

qualitatively shown in fig. 2, reflects the cascade through which correlations of 
more and more particles are created by successive collisions. Since two parti- 
cles that have already undergone a collision have little chance to collide again 
for large A^, due to the fact that the mean free path is much longer than the 
interparticle distance, information flows out from / (r, p) first to 2-particle cor- 
relations, then from 2-particle to 3-particle correlations, and so on. 

Figure 2 also exhibits a non-uniform convergence. If N is fixed, Sn tends to 
the constant S (D) at any time when n increases so as to reach A^. However, 
under physical conditions, we have always N ^ n, even if we let n ^> 1. In 
this case, Sn tends for any fixed n and for long times to the equilibrium entropy 
S^q. Thus, the amount of order S^q — S {D) which is in excess in the initial non- 
equilibrium state is eventually dissipated within the correlations of an infinite 
number of particles. This property is consistent with the way D (t) converges 
for large t towards the equilibrium state Deq of a perfect gas. Actually, for finite 
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Figure 2: Hierarchy of entropies in a gas. Larger and larger entropies are asso- 
ciated witli less and less detailed descriptions. The entropy 5" (D) of the complete 
description remains constant for an isolated system. If the initial state is uncorre- 
lated, Boltzmann's entropy Si increases (if-theorem) from S (D) to the equilibrium 
thermostatic entropy S^q, evaluated for the initial values of the energy, particle number 
and density of the gas. It is smaller than the thermodynamic entropy S'th, a function of 
the local densities of energy, particles and momentum, but both remain nearly equal 
after the distribution / (r, p, t) has become Maxwellian (local equilibrium regime). 
The entropies S2, S3 • • ■ Sn accounting for 2-, 3-,- • • n-particle correlations evolve like 
Si; they are at each time smaller and smaller with n and increase later and later. 

N, D {t) cannot reach Dgq since their entropies differ. However, if A'' is made 
large first, D {t) becomes equivalent for large t to Deq as regards / (r, p) (a 
Maxwellian distribution) and as regards all the correlations of a finite number 
n of particles (these correlations are negligible). 

Note that the entropies of the above hierarchy are purely theoretical objects. 
The only ones that experiments may reach, more or less indirectly, are ^b, Sth 
and ^eq, depending on the circumstances. 

11 Spin echoes 

Let us return to the paradox of irreversibility. Consider a gas, enclosed in a 
vessel and prepared at the initial time to = in an off-equilibrium state (for 
instance with a non-uniform density). At the time r, it has reached equilibrium. 
If we were able to reverse suddenly the velocities of all the particles, we would 
recover the initial state at the time 2t. Such a violation of the arrow of time 
in thermodynamics cannot be experimentally achieved, because the needed in- 
version of the individual velocities is not feasible. Anyhow, since the number 
of microscopic configurations associated with the initial non-equilibrium state 
is enormously smaller than the number of those associated with the final equi- 
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librium state, the above hypothetical reversal of velocities should be performed 
rigorously, which is unthinkable; a small deviation would lead at the final time 
2r to a configuration having equilibrium features at the macroscopic scale, in 
contrast to the initial off-equilibrium configuration. 

Nevertheless, similar paradoxical evolutions which violate at a macroscopic 
scale the laws of thermodynamics have been built up, through subtle experi- 
ments of nuclear magnetism (Abragam and Goldman, 1982). The nuclear spins 
Si (i = 1, 2, . . . , N), the value s of which is a half-integer, lie in an external 
magnetic field permanently applied along the z-axis. They undergo a Larmor 
precession around this axis with angular frequency u^q along z and proportional 
to the field. The only quantities that can be observed in practice are the com- 
ponents of the total magnetic moment M (i) , equal in dimensionless units to 
the expectation value over the density operator D (t) of the total spin 

M = ^s, . (28) 

i 

One can act on the system only by means of time-dependent external fields cou- 
pled to M. One thus handles simultaneously all the spins, which globally rotate. 
In particular, application at time i of a brief magnetic pulse along some axis 
during a controlled delay allows one to suddenly perform any overall rotation, 
with a given angle Q around a given direction ^l/Q. This pulse modifies the 

density operator D (t) into UD (t) , where U = exp (ifl ■ Mj is the unitary 
transformation describing the considered rotation. 
The Hamiltonian has the following form 

H = huo M, + ^ hSuSi ■ s, + V;^ + Vs\ . (29) 

i 

Its first term generates the Larmor precession dWl/dt = oiq x M around z. The 
next terms are responsible for three different mechanisms of relaxation of M. 

(i) The applied field is not quite uniform. Each spin (s^), depending on its 
location, undergoes a Larmor precession with angular frequency ujq + SuJi. If 
all the spins (s^) lie initially along the x-axis, they precess at different speeds, 
nearly in the a::y-plane, and thus get graduately out of phase. Hence their sum 
M (t) spirals down to zero in the vicinity of the xy-plane. If we denote as d 
the statistical fluctuation of Sujiz (the average over i of which vanishes) the 
relaxation time for this process is 1/6. 

(ii) The spin-spin contribution Vss to (|29|l is the sum of the two-body dipolar 
interactions. Under the experimental conditions where luq is sufliciently large, 
the part of V^s which does not commute with is ineffective. These pairwise 
interactions play for the spins the same role as the interparticle interactions for 
a gas. A "flip-flop" process, which changes the configuration of a pair into 
It, is the equivalent of a collision, which changes the momenta of two particles 
in a gas. This mechanism gradually creates correlations between two, then 
three spins, and so on. By a process similar to the one described at the end of 
section 10, the state of the A^-spin system thus tends to a maximum entropy 



25 



Boltzmann-Gibbs distribution Dcq oc cxp 




apart from invisible many- 



spin correlations. The spin temperature depends only on the initial energy 
of the spins. The relaxation time for this process is traditionally denoted as T2. 

(iii) The spin-lattice contribution V^i to (|29|l couples the nuclear spins to 
the other degrees of freedom (the "lattice"), such as the phonons in a solid. It 
tends to thermalize the nuclear spins, imposing them the lattice temperature. 
Its associated relaxation time Ti is much longer than the duration of the spin 
echo experiments, owing to the weakness of the coupling between the nuclear 
spins and the other variables of the material. We shall thus disregard this term. 

The oldest type of spin echoes was discovered by Hahn soon after the birth 
of nuclear magnetic resonance. Such spin echo experiments are performed on 
liquids, the disorder of which smoothes out the interactions Vss- The Hamilto- 
nian thus reduces to the first two Zeeman terms of H29|l . One starts from an 
equilibrium state, at a sufficiently low temperature so that all spins are prac- 
tically oriented along z. By a pulse tt/2 applied along y just before the time 
ta — 0, one generates an off-equilibrium pure initial state D (0) with magne- 
tization Mx (0) having the largest possible value Ns, and My = Mz = 0. As 
indicated above under (i), the magnetization M (i) relaxes under the effect of 
the heterogeneity of the field. After a number of turns larger than at some 

time T larger than \/5 (but smaller than T2 and Ti), all three components of M 
have reached 0. If we imagine we have followed the motion at the microscopic 
scale, the state D (t) has remained pure: each one of the N vectors (si) has 
kept the maximum length s; its component along z is negligible as 5/ljq, while 
it points out in some direction in the xy-plane. This direction is determined by 
the local magnetic field, i.e., by Sujiz- Nothing seems to distinguish this state 
from the reduced state Z?r which describes thermodynamic equilibrium at large 
temperature. While the entropy S [D] has kept its initial value 0, the rele- 



vant entropy S [DkJ = <5'r (M = 0) associated with M has reached the largest 

possible value In (2s -t- 1). The system seems "dead" since only M = is 
observed in practice. 

However, superimposed to the permanent magnetic field, a pulse tt along 
X is applied at the time r. The components {six), (siy), and (siz) — are 
quasi suddenly changed by the corresponding unitary transformation into (six), 
— (siy), and — (siz) — 0. Hence, the spins which precess faster than the average 
Larmor flow because Sujiz > 0, and which were therefore ahead at the time 
r by some angle, are now behind at the time r + by just the same angle. 
Likewise, the spins that precess slower have been brought forward by the pulse. 
Hence, after a second time lapse t, during which the precession goes on with the 
same local frequency, all the spins reach the initial direction along x, and the 
total magnetization M returns to its largest possible value Ns at the time 2r. 
Between the times t and 2t, the initial order has progressively been recovered, 
the relevant entropy S'r, (M) has decreased from A^ln(2s-f 1) to 0. A single 
macroscopic manipulation, the application of the pulse tt along x at r, has been 
sufficient to produce an evolution which violates the thermodynamic surmise, as 
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would have done the unfeasible reversal of velocities for a gas. This paradoxical 
evolution can be understood only by keeping track of all the microscopic spin 
degrees of freedom. 

In this spin echo experiment, the order about M, lost between the time 
and T and recovered between r and 2r, is associated with the correlation 
for each spin, which during the motion is established between the direction 
of the vector (s^) in the .xy-plane and the value Sujiz of the z-componcnt of 
the local magnetic field. The very large number ojot/H of revolutions of the 
spins that take place during the time r produces a configuration of directions 
of spins (§,;) which seems to be just a sample picked up at random within the 
truly dead ensemble -Dr describing equilibrium. Even if we are given the full 
available information, to wit, the seemingly dead density operator D (r) issued 
from D (0) and the Hamiltonian H that generated the evolution, we would 
have an extreme difficulty to uncover, among this huge amount of data, the 
crucial correlations between (sj) and Suiiz- Our situation is the same as that of 
the audience of a conjuring show, who are unable to detect how the shuffling 
process has retained some hidden order in a pack of cards. The experimentalist 
has also the same pieces of information, D (r) and H, but like the conjurer he 
acknowledges therein that the directions of the (si)'s are not random but are 
correlated with the ^Wj^'s. Indeed, he is aware of the initial conditions and of the 
history of the system which destroyed the visible order, an information which is 
equivalent to the knowledge of the correlations between D (t) and H, but easier 
to analyse. Relying on this knowledge, he is able to bring back the order buried 
in these correlations into the simple macroscopic degrees of freedom M. 

He succeeds in this task owing to a specificity of the motion: it is two- 
dimensional except during a pulse, and we can see that the rotation U around 
the X-axis has an effect akin to a time reversal. Indeed, noting that the final state 
is invariant under the rotation U = around x, we can write the evolution 
operator between the times and 2r as 

W exp (^-iHr/hj Uexp (^-iHr/hj = exp (^-iW HUr/hj exp (-iHr/h) . 

(30) 

Since the components ojix of the field have little effect, we can write 

U'^HU -H . (31) 

This change of sign has the same effect as a time-reversal taking place during 
the second part of the process, which therefore brings us back to the initial 
states at the time 2r. 

After the time 2t, the evolution is exactly the same as after the initial 
time and M {t) returns to zero. The operation can then be iterated, and the 
measurement of the length of M {t) provides a sequence of peaks. Their height 
slowly decreases due to the other relaxation mechanisms that we have neglected 
on the time scale 1/6. Spin echoes have thus become a current technique in 
nuclear magnetic resonance to determine relaxation times with precision and 
thus explore matter. 
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At the epoch of their discovery, these spin echoes were regarded as some- 
what miraculous since they violated thermodynamics: They appeared as an 
exceptional macroscopic phenomenon that can be explained only through the 
microscopic dynamics. Full knowledge of D (r) and of H is needed here, whereas 
the thermodynamic phenomena can be described in terms of the reduced state 
Dth and of H only. We may, however, argue that the order which is retrieved was 
not very deeply hidden. It lay at the time r in the rather trivial set of correla- 
tions between the orientation of each spin and the size of the corresponding local 
field. It is natural to think that the relaxation induced by the spin-spin interac- 
tions should in actual fact be incurable, in the same way as the collision-induced 
relaxation in a gas. Nevertheless, quite surprisingly, spin-echo experiments of 
a different type have been performed, in which the "death" of spins (M 0) 
caused after the relaxation time T2 by the term Vss of (|29|l is followed by their 
"resurrection" (Abragam and Goldman, 1982). We only sketch here the ba- 
sic ideas. The considered experiments are performed on solids, for which Vss 
is significant. The applied field is sufficiently uniform so that the duration of 
the experiment is much shorter than l/S. The Hamiltonian (|29|l can therefore 
be simplified into H — huj^Mz -f Vss- The experiment begins as in Hahn's spin 
echoes by the preparation of an initial state with = Ns, My = = 0, then 
during the Larmor precession by its decay, taking now place under the effect of 
t^s- At a time r larger than T2, the magnetization is lost as above, but now the 
relaxation has taken place (as in the case of gases) through a cascade of flip-flop 
transitions involving more and more spins. The initial order has dissolved into 
complicated many-spin correlations, and the density operator 

i) (r) = e^'^'-Z^D (0) e'"^/^ (32) 

is equivalent, as regards observables involving a finite number of spins, to a 
"dead" equilibrium state with large temperature. 

After the time r, a sequence of suitably chosen brief pulses and lasting time- 
dependent fields referred to as a "magic sandwich" , is applied for a duration 
t'. Thus, during this period, the Hamiltonian becomes H + hu) (t) ■ M where 
u) (t) is proportional to the time-dependent applied field. The remarkable fact is 
the possibility of choosing a; {t) in such a way that the unitary operator which 
describes the evolution between the times r and r -f t' has approximately the 
form 

Texp|-i J dt H/h + uj{t) -M |~exp iXHr'/h (33) 

(T denotes time-ordering). The coefficient A is positive, so that, as in eqs. 
H3UI) (|31|l . the effect of the magic sandwich u) (t) is to replace r' by —At', thus 
changing effectively the arrow of time in the considered interval. The actual re- 
alization of eq. has needed great ingenuity, because the dipolar interactions 
contained in H involve pairwise interactions whereas the external operations are 
represented by a mere field coupled to M only. 

As an example, in one of the experiments performed, u) (t) in 1)3311 describes 
a magic sandwich consisting of three successive operations: (i) at the time t. 
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a pulse 7r/2 along y is exerted, represented by the operator U ; (ii) between the 
times T and r + r', a radiofrequency field along x, with a frequency ojq and an 
intensity hu' is applied; in the Larmor rotating frame, it provides a contribution 
hu'Mx to the Hamiltonian; (iii) at the time r + r' a pulse —tt/2 is apphed along 
y. The evolution operator written in the rotating frame, is thus 



exp 



i (v;s/?i + w'M:r) t' [/ = exp(^-i[/^V;s?7T7;i-iw'Af^r') . (34) 



If Lu' is sufficiently large, the part of U^VssU which does not commute with 
is negligible, and one finds 

WVssU --^Vss, (35) 

which replaces here (j31|l . The last term in H34|l amounts to a trivial Larmor 
rotation. Thus (|33|l is satisfied (in the rotating frame) with A = ^. 

After the time r + r' the evolution is again governed by the sole Hamiltonian 
H. Hence, provided r' is larger than r/A, we can introduce a delay r" such 
that T — Ar' + t" = 0, for which the full evolution operator from i = to 
t = T + t' + t" , found from l|32|l and H33|l is nearly the unit operator. The 
magnetization M (r + r' + r") thus returns to its initial value M (0) although 
it had previously decayed to zero during the time r. Here again the arrow of 
time is challenged. 

The features of this process are the same as in Hahn's echoes, in particular 
the recovery of hidden order and the need for explaining this phenomenon to 
rely on all the microscopic degrees of freedom^ although the observations and 
the manipulations concern only the macroscopic quantities M. However, here, 
the order that is experimentally retrieved had been transferred by the evolution 
towards extremely complicated correlations of large numbers of spins, and the 
initial decay seemed genuinely irreversible as in the case of a gas. Indeed, for 
nearly all experiments which can be realized in practice, the state D (r) which 
involves these special correlations keeping memory of D (0) gives the same pre- 
dictions as the reduced equilibrium state I)r. Nevertheless, the magic sandwich 
manipulation, which is rather tricky, transforms back these special correlations 
of D (r) into the initial order restored in D (r + r' + r"). The magic sandwich 
experiments achieve, for spins, the analogue of the hypothetical reversal of ve- 
locities for a gas; quite remarkably, it is simply the application of some special 
uniform time-dependent magnetic fields which overcomes here the apparent dis- 
order generated by the interactions. 



12 Conclusion 

An analysis of various physical processes where we made use of the informa- 
tion concept has led us to stress the relative nature of entropy, identified with 
missing information. Depending on the circumstances, macroscopic phenomena 
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should be described in terms of different sets of relevant variables. Different as- 
sociated entropies are thus introduced, even for the same system. The entropy 
of thermodynamics is only one of them. 

Dissipation, which is also relative, is measured by the increase of some rel- 
evant entropy. It characterizes the irreversibility of a macroscopic process, and 
can be interpreted as an irretrievable leak of information towards inaccessible 
degrees of freedom. Since information, as well as probability, is a concept as- 
sociated with the knowledge of observers about an object rather than with the 
object in itself, thermodynamic notions such as entropy and dissipation have at 
the microscopic level a subjective aspect. 

This subjective character is exemplified by the spin-echo experiments, which 
show that irreversibility itself is relative. In such circumstances the loss of 
memory which accompanies an irreversibility may be overcome and a hidden 
order, which keeps track of the initial state, may come into view. 

However, in less exotic circumstances, for instance, for systems displaying 
a collective motion or for materials that can be described by thermodynamics, 
it is possible to select among the huge set of microscopic variables some subset 
of relevant variables which obey autonomous equations of motion. The lat- 
ter dynamical equations are often identified with phenomenological equations. 
Their establishment from microphysics involves approximations, such as the 
short-memory approximation, but they can hold with a high precision if there 
is a clear-cut separation of time-scales which allows the elimination of irrele- 
vant variables. The existence of these variables manifests itself through quasi- 
instantaneous dissipative effects. Such a reduction of the description brings in 
new features, which differ qualitatively from those of microphysics, especially 
for macroscopic systems: continuity and extensivity of matter at our scale, ir- 
reversibility and non-linearity of the equations of motion, existence of phase 
transitions, enormous variety of behaviours in spite of the simplicity of the ele- 
mentary constituents and in spite of the unicity of the microscopic Laws. Due 
to the change of scale, the statistical fluctuations of the relevant observables are 
small compared to experimental or computational errors. The very nature of 
our description is thus changed. Although the underlying, more fundamental 
description yields predictions having subjective features due to the necessary use 
of probabilities, it gives way to a reduced description that no longer involves 
the observer. Since the variances are negligible, the physical quantities do not 



obey the reduced dynamics can be interpreted merely as values actually taken 
in a given system. Their equations of motion then directly apply to individual 
objects, without reference to a statistical ensemble. In spite of the disappear- 
ance of probabilities at the macroscopic scale, the relevant entropy survives as 
a quantity measurable indirectly through its variations and characterizing 
equilibrium, as well known in thermodynamics. Its probabilistic and subjec- 
tive origin, its interpretation as missing information arc not apparent; neither 
are they for the other variables, the expectations Ai and the parameters 7^ of 
the microscopic probability law. Actually all these quantities had already been 



need to be regarded as random variables. The expectation values 
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introduced in thermodynamics at the macroscopic level, but the advent of sta- 
tistical mechanics gave us a deeper understanding. Indeed, statistical mechanics 
is a major tool of reductionism; its use explains the emergence at a larger scale 
of new properties, new concepts and even different interpretations of physical 
statements, such as here the emergence of objectivity from a microscopic theory 
based on the subjective concepts of probability and information. Likewise, treat- 
ing quantum measurements as a problem of statistical mechanics (AUahvcrdyan 
et al, 2003) shows how ordinary probabilities emerge from the underlying irre- 
ducible extension of probabilities which accounts for the non-commutation of 
quantum observables. 

Let us finally recall that the identification of entropy with a lack of infor- 
mation or equivalently with disorder has contributed to the elucidation of the 
paradox of Maxwell's demon, which gave rise to discussions for more than one 
century. An important step was the recognition of the equivalence between 
information and negentropy (Brillouin, 1956). The entropy of a system spon- 
taneously increases; however it may be lowered by some amount, provided the 
"demon" or the experimentalist who acts upon this system makes use of some 
amount of information at least equal to the decrease of the entropy. Conversely, 
in order to acquire some information, we need to raise the entropy of some 
system by a quantity at least equal to the amount of information gained. Infor- 
mation theory thus enlightens many aspects of statistical physics. 

The above text is issued from a talk given at the ESF conference on philosophical 
and foundational issues in statistical physics, held in Utrecht on 28-30 November 2003. 
1 wish to thank Jos Uffink for his invitation to this conference and for comments. 
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